1. Field of the Invention
The present invention relates to an image input method which obtains an image having a broad range by compositing a plurality of images, to an image input apparatus, and to a recording medium which stores programs for realizing the functions of this image input apparatus and which is computer-readable.
2. Background Art
Image input machines which are commonly employed, such as video cameras or the like, have a limited field of view, so that they are thought to produce images of a small part of the information of the outside world surrounding a user. When image input apparatuses are employed which are capable of obtaining images having a wide field of view in which this limited field of view is expanded, for example in the observation of the interior or exterior of a store, conventionally, a watchman was only able to view images in his immediate vicinity, whereas with these images having a wider field of view, the state of the surroundings may also be assessed quickly, so that the ability to confirm visually the state of the surroundings is dramatically improved. Furthermore, when image input devices having a wide field of view are employed, the range which may be observed by a single device is increased, and this is expected to lead to a general decrease in costs.
The main conventional image acquisition methods involve the scanning of film affixed to the side surface of a cylinder using light projected from slits moving over the opposite side surface (the so-called panorama camera) or the direct acquisition of wide-viewfield images using a fish-eye lens or a wide angle lens. The wide-viewfield images could be used as panorama pictures, or, in the case of fish-eye lenses or the like, could be passed through an optical system conducting a reverse conversion and then projected onto a screen; in order to employ the wide-viewfield images in a computer, the film or prints had to be inputted using a scanner, and this involved time and labor.
As it became possible to directly input images into a computer as a result of advances in hardware, by applying certain conversions or compositing processing to the images inputted into a computer, it became possible to obtain wide-viewfield images. Methods for the acquisition of wide-viewfield images which were precursors to such processing in computers were broadly divided into image compositing types, in which partial images taken using a standard camera were composited to produce a wide-viewfield image, and optical types, in which a wide-viewfield image was directly acquired using a wide angle lens or a mirror.
The image compositing types could be further classified into methods in which a plurality of partial images are acquired using a tripod or universal head which is adjusted so that a camera rotates thereon, and these are then composited to produce a wide-viewfield image, and methods in which partial images obtained using a plurality of cameras are composited. In the former, because a plurality of images acquired using standard cameras are composited, the resulting wide-viewfield image has a high degree of resolution; however, as disclosed in Japanese Patent Application, First Publication No. HEI 9-18750, xe2x80x9cElectronic camera and electronic camera system employing this, photographic apparatus, signal processing apparatusxe2x80x9d, or Japanese Patent Application, First Publication No. HEI 8-116490, xe2x80x9cImage processing apparatusxe2x80x9d, in order to avoid the problem of different points of view which is caused between images as a result of movements in the point of view in the conventional method, a mechanism is necessary for rotating the camera about the principal point of the lens. Furthermore, there were problems with such a method in that the operation by which the partial images were acquired was complex, the camera was heavy and large-sized, and as a result of the characteristics of the signal and power source cables, the camera could not be rotated continuously and at high speed, so that it was difficult to continuously acquire a wide-viewfield image.
In the latter methods, in which partial images obtained using a plurality of cameras were composited, cameras were arranged in a fixed manner, so that once the correct setting had been achieved, it was possible to acquire images in real time with little delay, but as a large number of cameras had to be deployed, the size of the apparatus was large, and furthermore, because the cameras were fixed, it was difficult to alter the degree of resolution or the image acquisition range. Furthermore, when the number of cameras was increased, the mechanism by which the image of each camera was inputted into the computer became elaborate.
In the optical types, a wide-viewfield image was directly acquired using a wide angle lens or mirror, and then the image was reconstituted using certain conversions. Methods have been developed which employ a fish-eye lens and which employ a mirror. In either method, a lens or mirror which is capable of acquiring the wide-viewfield image in a single image acquisition is fixed, and real-time acquisition is possible; however, there is a problem in that the degree of resolution of the wide-viewfield image obtained is low. In particular, in methods which employ fish-eye lenses, large distortions are present at the edges of images, and extremely rough images are obtained even if certain conversions are employed. There are also methods in which wide-viewfield image input apparatuses employing mirrors are combined with cameras which require close observation point images; however, although the close observation point images have a high degree of resolution, this does not solve the problem of the insufficient degree of resolution of the wide-viewfield images themselves. A method which is capable of solving the problems in the conventional wide-viewfield image input methods, and which is capable of acquiring, continuously and at high speed, wide-viewfield images having a high degree of resolution, is a method in which a plurality of partial images acquired by rotating a mirror in front of a camera are composited. If changes in the actual vertical or left-right correspondence with the image, and geometrical distortions, are excluded, then the rotation of the mirror about the optical axis of the camera corresponds to a pan operation, and the rotation of the mirror about an axis perpendicular to the optical axis of the camera corresponds to a tilt operation of the camera, so that using the rotation of this mirror, a plurality of partial images are acquired, and the partial images obtained are subjected to certain conversions, are composited, and a wide-viewfield image is acquired.
However, as pointed out in the specification of Japanese Patent Application, First Publication No. HEI 9-18750 described above, the following problems are present in image input apparatuses which input wide-viewfield images by compositing partial images obtained by causing a conventional mirror to rotate:
1. The mirror which serves to switch the photographic field is large, and
2. Differences in viewpoint are produced when the photographic field is changed by the mirror which is disposed in front of the lens.
In particular, the latter problem is one which is also caused when the camera is rotated at a point other than the principal point of the lens, and this causes problems in that the continuity of the composite image is negatively affected, and the image quality of the wide-viewfield image worsens dramatically.
The present invention was created in light of the above circumstances; it has as a first object thereof to provide an image input method which, in the case in which a camera is rotated about a point other than the principal point of the lens, and during the compositing of a plurality of partial images obtained by rotating a mirror before the camera, makes it possible to reduce the effects of parallax contained in the partial images and to input a wide-viewfield image having high image quality, as well as to provide an image input apparatus and a computer-readable recording medium for storing programs for executing the functions of the image input apparatus.
Furthermore, it is a second object of the present invention to provide an image input apparatus which, when compositing a plurality of partial images obtained by rotating a mirror in front of a camera, makes it possible to input wide-viewfield images at high speed.
In order to achieve the first object described above, the invention according to the first aspect comprises an image input method for inputting a wide-viewfield composite image by photographing parts of a field to be photographed while changing the photographing direction by rotating a camera which is supported in a rotatable manner, projecting the plurality of partial images of the field to be photographed acquired in this manner onto a projection surface, and based on the results of this projection, conducting compositing on a composite image plane, wherein, during the compositing of the plurality of partial images, the center of rotation of the camera is used as the center of the projection, and the intersection point between the projection surface established in space and a straight line, which is parallel to a straight line connecting the viewpoint of the camera and a pixel in a partial image in an image plane formed by the plurality of partial images, and which passes through the rotational center of the camera, is employed as the projection point of the pixel in the partial image.
In accordance with the invention according to the first aspect, it is possible to obtain a satisfactory wide-viewfield composite image in which the effects of differences in view contained in the partial images are minimized, even when the camera is rotated about a point other than the principal point of the lens.
Furthermore, the invention according to the second aspect comprises an image input method for inputting a wide-viewfield composite image by photographing parts of a field to be photographed while altering the range of photography in the field to be photographed of a camera by rotating a mirror supported in a rotatable manner before the camera, projecting a plurality of partial images of the field to be photographed which were acquired onto a projection surface, and based on the results of this projection, conducting compositing on a composite image surface, wherein the rotational center of the mirror is made the center of projection during the compositing of the plurality of partial images, and the point of intersection between the projection surface which is established in space and a straight line, which is parallel to a reflection straight line indicating the reflected light ray which results when a light ray from the viewpoint of the camera which passes through a pixel of a partial image in an image plane formed by the partial images is reflected by the mirror, and which passes through the rotational center of the mirror, is employed as the projection point of the partial image.
In accordance with the invention according to the second aspect, satisfactory wide-viewfield composite images, in which the effects of differences in viewpoint incorporated in the partial images are limited, may be obtained.
The invention according to the third aspect comprises an image input method for inputting wide-viewfield composite images by photographing parts of a field to be photographed while altering the direction of photography by rotating a camera supported in a rotatable manner, and compositing the plurality of partial images of the field to be photographed which are acquired, wherein a pixel on a composite image surface corresponding to a point on a projection surface is determined from an intersection point between an image plane formed by the plurality of partial images acquired and a straight line, which is parallel to a straight line connecting the point on the image plane established in space at the time of the compositing of the plurality of partial images and the rotational center of the camera, and which passes through the viewpoint of the camera.
In accordance with the invention according to the third aspect, in addition to the effects achieved by the invention according to the first aspect, an effect is achieved whereby, when determining pixel values in the composite image by means of, for example, arithmetic processing such as linear interpolation or the like, blank pixels, which are a product of rounding errors in the coordinate values indicating the positions of the pixels, are not produced in the composite image.
The invention according to the fourth aspect comprises an image input method for inputting a wide-viewfield composite image by photographing parts of a field to be photographed while altering the photographic range in the field to be photographed of a camera by rotating a mirror which is supported in a rotatable manner before the camera, and compositing a plurality of partial images of the field to be photographed which are obtained, wherein a pixel on a composite image surface corresponding to a point in a projection surface is determined from an intersection point between an image plane formed by the partial images and a straight line, which is parallel to a straight line indicating a reflected light beam resulting from a reflection by the mirror of a straight line connecting a point on the projection surface which is established in space at the time of the compositing of the plurality of partial images and the rotational center of the mirror, and which passes through the viewpoint of the camera.
In accordance with the invention according to the fourth aspect, in addition to the effects of the invention according to the second aspect, when the pixel values in the composite image are determined by means of, for example, arithmetic processing such as linear interpolation or the like, blank pixels, which are a product of rounding errors in the coordinate values indicating the position of the pixels, are not produced in the composite image.
The invention according to the fifth aspect comprises an image input method for inputting wide-viewfield composite images by photographing parts of a field to be photographed while altering the photographic direction by rotating a camera which is supported in a rotatable manner, projecting a plurality of partial images of the field to be photographed which are obtained onto a projection surface, and based on the results of this projection, conducting compositing on a composite image surface, wherein is provided an image compositing mechanism, which employs the rotational center of the camera as the center of projection during the compositing of the plurality of partial images, and which employs, as the projection point of a pixel of a partial image, the intersection point between the projection surface which is established in space and a straight line, which is parallel to a straight line connecting the viewpoint of the camera and the pixel in the partial image in an image plane formed by the plurality of partial images, and which passes through the rotational center of the camera.
Furthermore, the invention according to the sixth aspect comprises an image input apparatus for inputting wide-viewfield composite images by photographing parts of a field to be photographed while altering the photographic direction by rotating a camera mounted in a rotatable manner, projecting a plurality of partial images of the field to be photographed obtained into a projection surface, and based on the results of this projection, conducting compositing on a composite image surface, wherein are provided: a camera movement mechanism for changing the photographic direction of the camera; a posture control mechanism for directing the camera movement mechanism so as to change the photographic direction of the camera and for transmitting camera posture data to all parts; a first storage mechanism for storing partial images and composite images produced by compositing the partial images; a second storage mechanism for storing attribute data related to partial images and composite images and camera posture data from partial image acquisition; a partial image acquisition mechanism for incorporating images photographed by the camera and acquiring partial images from these incorporated images, and storing the partial images in the first storage mechanism, and for storing posture data from the acquisition of the partial images of the camera received from the posture control mechanism, and attribute data relating to the partial images and composite images, in the second storage mechanism; and an image compositing mechanism for setting the rotational center of the camera as a center of projection, determining a projection point so that an intersection point between a projection surface established in space and a straight line, which is parallel with a straight line connecting the viewpoint of the camera and a pixel in a partial image in an image plane formed by the plurality of partial images, and which passes through the rotational center of the camera, is employed as the projection point of the pixel in the partial image, compositing the partial images with reference to the contents of the first and second storage mechanisms, and storing a composite image in the first storage mechanism.
In accordance with the invention according to the fifth and sixth aspects, it is possible to obtain a satisfactory wide-viewfield composite image, in which the effects of differences in viewpoint contained in the partial images are reduced, even when the camera is rotated about a point other than the principal point of the lens.
Furthermore, the invention according to the seventh aspect comprises an image input apparatus for inputting wide-viewfield composite images by photographing parts of a field to be photographed while altering the photographic direction by rotating a camera mounted in a rotatable manner, projecting a plurality of partial images of the field to be photographed obtained into a projection surface, and based on the results of this projection, conducting compositing on a composite image surface, wherein are provided: a camera movement mechanism for changing the photographic direction of the camera; a posture control mechanism for directing the camera movement mechanism so as to change the photographic direction of the camera and for transmitting camera posture data to all parts; a first storage mechanism for storing partial images and composite images produced by compositing the partial images; a second storage mechanism for storing attribute data related to partial images and composite images and camera posture data from partial image acquisition; a partial image acquisition mechanism for incorporating images photographed by the camera and acquiring partial images from these incorporated images, and storing the partial images in the first storage mechanism, and for storing posture data from the acquisition of the partial images of the camera received from the posture control mechanism, and attribute data relating to the partial images and composite images, in the second storage mechanism; a positioning mechanism for determining positional relationships between partial images stored in the first storage mechanism, determining positional correction amounts of the composited positions of the partial images stored in the second storage mechanism based on the positional relationships obtained, and storing these in the second storage mechanism; and an image compositing mechanism for determining compositing positions of the partial images using the camera posture data from the time at which the partial images were obtained, and the positional correction amounts of the compositing positions obtained by the positioning mechanism, and in accordance with the compositing positions obtained, setting the rotational center of the camera as a center of projection, determining a projection point so that an intersection point between a projection surface established in space and a straight line, which is parallel with a straight line connecting the viewpoint of the camera and a pixel in a partial image in an image plane formed by the plurality of partial images, and which passes through the rotational center of the camera, is employed as the projection point of the pixel in the partial image, compositing the partial images with reference to the contents of the first and second storage mechanisms, and storing a composite image in the first storage mechanism.
In accordance with the invention according to the seventh aspect, it is possible to obtain a satisfactory wide-viewfield composite image, in which the effects of differences in viewpoint contained in the partial images are reduced, even when the camera is rotated about a point other than the principal point of the lens, and it is possible to produce a high-precision composite image in which errors in the angle detection mechanism of the camera which comprises the camera movement mechanism are reduced.
By means of appropriately setting the existing range of camera orientation during the acquisition of partial images, it is possible to omit the angle detection mechanism of the camera which comprises the camera movement mechanism in the structure of the image input apparatus.
The invention according to the eighth aspect comprises an image input apparatus for inputting a wide-viewfield composite image by photographing parts of a field to be photographed while changing the range of photography in the field to be photographed of a camera by rotating a mirror which is supported in a rotatable manner before the camera, projecting a plurality of partial images of the field to be photographed which are obtained onto a projection surface, and based on the results of this projection, conducting compositing on a composite image surface, wherein an image compositing mechanism is provided which uses the rotational center of the mirror as the center of projection during the compositing of the plurality of partial images, and which employs the intersection point between the projection surface which is established in space and a straight line, which is parallel to a reflection straight line indicating a reflected beam of light resulting from the reflection by the mirror of a light beam from the viewpoint of the camera which passes through a pixel of a partial image in an image plane formed by the partial images, and which passes through the rotational center of the mirror, as the projection point of the partial image.
The invention according to the ninth aspect comprises an image input apparatus for inputting a wide-viewfield composite image by photographing parts of a field to be photographed while changing the range of photography in the field to be photographed of a camera by rotating a mirror which is supported in a rotatable manner before the camera, projecting a plurality of partial images of the field to be photographed which are obtained onto a projection surface, and based on the results of this projection, conducting compositing on a composite image surface, wherein are provided: a mirror movement mechanism for altering the direction of the mirror so as to change the photographic direction of the camera; a mirror control mechanism for directing the mirror movement mechanism so as to change the orientation of the mirror and for transmitting posture data of the mirror to all parts; a first storage mechanism for storing partial images and composite images produced by compositing the partial images; a second storage mechanism for storing attribute data related to partial images and composite images and the mirror posture data from partial image acquisition; a partial image acquisition mechanism for incorporating images photographed by the camera and acquiring partial images from these incorporated images, and storing the partial images in the first storage mechanism, and for storing posture data from the acquisition of the partial images of the mirror received from the mirror control mechanism, and attribute data relating to the partial images and composite images, in the second storage mechanism; and an image compositing mechanism which uses the rotational center of the mirror as the center of projection during the compositing of the plurality of partial images, and which determines a projection point such that the intersection point between the projection surface which is established in space and a straight line, which is parallel to a reflection straight line indicating a reflected beam of light resulting from the reflection by the mirror of a light beam from the viewpoint of the camera which passes through a pixel of a partial image in an image plane formed by the partial images, and which passes through the rotational center of the mirror, is employed as the projection point of the pixel of the partial image, and which composites the partial images with reference to the contents of the first and second storage mechanisms, and stores a composite image in the first storage mechanism.
In accordance with the invention according to the eighth and ninth aspects, it is possible to obtain satisfactory wide-viewfield composite images in which the effects of differences in viewpoint contained in the partial images are reduced.
The invention according to the tenth aspect comprises an image input apparatus for inputting a wide-viewfield composite image by photographing parts of a field to be photographed while changing the range of photography in the field to be photographed of a camera by rotating a mirror which is supported in a rotatable manner before the camera, projecting a plurality of partial images of the field to be photographed which are obtained onto a projection surface, and based on the results of this projection, conducting compositing on a composite image surface, wherein are provided: a mirror movement mechanism for altering the direction of the mirror so as to change the photographic direction of the camera; a mirror control mechanism for directing the mirror movement mechanism so as to change the orientation of the mirror and for transmitting posture data of the mirror to all parts; a first storage mechanism for storing partial images and composite images produced by compositing the partial images; a second storage mechanism for storing attribute data related to partial images and composite images and the mirror posture data from partial image acquisition; a partial image acquisition mechanism for incorporating images photographed by the camera and acquiring partial images from these incorporated images, and storing the partial images in the first storage mechanism, and for storing posture data from the acquisition of the partial images of the mirror received from the mirror control mechanism, and attribute data relating to the partial images and composite images, in the second storage mechanism; a positioning mechanism for determining positional relationships between partial images stored in the first storage mechanism, determining positional correction amounts of the composited positions of the partial images stored in the second storage mechanism based on the positional relationships obtained, and storing these in the second storage mechanism; and an image compositing mechanism for determining compositing positions of the partial images using the camera posture data from the time at which the partial images were obtained, and the positional correction amounts of the compositing positions obtained by the positioning mechanism, and in accordance with the compositing positions obtained, setting the rotational center of the mirror as the center of projection during the compositing of the plurality of partial images, determining a projection point such that the intersection point between the projection surface which is established in space and a straight line, which is parallel to a reflection straight line indicating a reflected beam of light resulting from the reflection by the mirror of a light beam from the viewpoint of the camera which passes through a pixel of a partial image in an image plane formed by the partial images, and which passes through the rotational center of the mirror, is employed as the projection point of the pixel of the partial image, compositing the partial images with reference to the contents of the first and second storage mechanisms, and storing a composite image in the first storage mechanism.
In accordance with the invention according to the tenth aspect, it is possible to obtain satisfactory wide-viewfield composite images in which the effects of differences in viewpoint contained in the partial images are reduced, and it is possible to produce highly precise composite images in which errors in the angle detection mechanism of the mirror which constitutes the mirror movement mechanism are reduced.
Furthermore, by means of appropriately setting the existing range of mirror orientation during the acquisition of partial images, it is possible to omit the angle detection mechanism of the mirror which comprises the mirror movement mechanism in the structure of the image input apparatus.
The invention according to the eleventh aspect comprises an image input apparatus for inputting wide-viewfield composite images by photographing parts of a field to be photographed while altering the direction of photography by rotating a camera supported in a rotatable manner, and compositing the plurality of partial images of the field to be photographed which are acquired, wherein is provided an image compositing mechanism for determining a pixel on a composite image surface corresponding to a point on a projection surface from an intersection point between an image plane formed by the plurality of partial images acquired and a straight line, which is parallel to a straight line connecting the point on the image plane established in space at the time of the compositing of the plurality of partial images and the rotational center of the camera, and which passes through the viewpoint of the camera.
The invention according to the twelfth aspect comprises an image input apparatus for inputting wide-viewfield composite images by photographing parts of a field to be photographed while altering the direction of photography by rotating a camera supported in a rotatable manner, and compositing the plurality of partial images of the field to be photographed which are acquired, wherein are provided: a camera movement mechanism for changing the photographic direction of the camera; a posture control mechanism for directing the camera movement mechanism so as to change the photographic direction of the camera and for transmitting camera posture data to all parts; a first storage mechanism for storing partial images and composite images produced by compositing the partial images; a second storage mechanism for storing attribute data related to partial images and composite images and camera posture data from partial image acquisition; a partial image acquisition mechanism for incorporating images photographed by the camera and acquiring partial images from these incorporated images, and storing the partial images in the first storage mechanism, and for storing posture data from the acquisition of the partial images of the camera received from the posture control mechanism, and attribute data relating to the partial images and composite images, in the second storage mechanism; and an image compositing mechanism for determining a pixel on a composite image surface corresponding to a point on the projection surface from the intersection point between an image plane formed by the plurality of partial images obtained and a straight line, which is parallel to a straight line connecting a point in the projection surface established in space at the time of the compositing of the plurality of partial images with the rotational center of the camera, and which passes through the viewpoint of the camera, compositing the partial images with reference to the contents of the first and second storage mechanisms, and storing the composite image in the first storage mechanism.
In accordance with the invention according to the eleventh and twelfth aspects, even when the camera is rotated about a point other than the principal point of the lens, it is possible to obtain a satisfactory wide-viewfield composite image in which the effects of differences in viewpoint contained in the partial images are reduced, and an effect is achieved whereby, when determining the pixel values in the composite image by means of arithmetic processing such as linear interpolation or the like, blank pixels, which are a product of rounding errors in the coordinate values indicating the positions of the pixels, are not produced in the composite image.
The invention according to the thirteenth aspect comprises an image input apparatus for inputting wide-viewfield composite images by photographing parts of a field to be photographed while altering the direction of photography by rotating a camera supported in a rotatable manner, and compositing the plurality of partial images of the field to be photographed which are acquired, wherein are provided: a camera movement mechanism for changing the photographic direction of the camera; a posture control mechanism for directing the camera movement mechanism so as to change the photographic direction of the camera and for transmitting camera posture data to all parts; a first storage mechanism for storing partial images and composite images produced by compositing the partial images; a second storage mechanism for storing attribute data related to partial images and composite images and camera posture data from partial image acquisition; a partial image acquisition mechanism for incorporating images photographed by the camera and acquiring partial images from these incorporated images, and storing the partial images in the first storage mechanism, and for storing posture data from the acquisition of the partial images of the camera received from the posture control mechanism, and attribute data relating to the partial images and composite images, in the second storage mechanism; a positioning mechanism for determining positional relationships between partial images stored in the first storage mechanism, determining positional correction amounts of the composited positions of the partial images stored in the second storage mechanism based on the positional relationships obtained, and storing these in the second storage mechanism; and an image compositing mechanism for determining compositing positions of the partial images using the camera posture data from the time at which the partial images were obtained, and the positional correction amounts of the compositing positions obtained by the positioning mechanism, and in accordance with the compositing positions obtained, determining a pixel on a composite image surface corresponding to a point on the projection surface from the intersection point between an image plane formed by the plurality of partial images obtained and a straight line, which is parallel to a straight line connecting a point in the projection surface established in space with the rotational center of the camera, and which passes through the viewpoint of the camera, compositing the partial images with reference to the contents of the first and second storage mechanisms, and storing the composite image in the first storage mechanism.
In accordance with the invention according to the thirteenth aspect, even when the camera is rotated about a point other than the principal point of the lens, it is possible to obtain a satisfactory wide-viewfield composite image in which the effects of differences in viewpoint contained in the partial images are reduced, and an effect is achieved whereby, when determining the pixel values in the composite image by means of arithmetic processing such as linear interpolation or the like, blank pixels, which are a product of rounding errors in the coordinate values indicating the positions of the pixels, are not produced in the composite image.
Furthermore, it is possible to reduce the errors of the angle detection mechanism of the camera which constitutes the camera movement mechanism, and to produce highly accurate composite images.
Additionally, by appropriately setting the existing range of orientation of the camera during the acquisition of the partial images, it is possible to omit the angle detection mechanism of the camera, which comprises the camera movement mechanism in the structure of the image input apparatus.
The invention according to the fourteenth aspect comprises an image input apparatus for inputting a wide-viewfield composite image by photographing parts of a field to be photographed while altering the photographic range in the field to be photographed of a camera by rotating a mirror which is supported in a rotatable manner before the camera, and compositing a plurality of partial images of the field to be photographed which are obtained, wherein is provided an image compositing mechanism for determining a pixel on a composite image surface corresponding to a point in a projection surface from an intersection point between an image plane formed by the partial images and a straight line, which is parallel to a straight line indicating a reflected light beam resulting from a reflection by the mirror of a straight line connecting a point on the projection surface which is established in space at the time of the compositing of the plurality of partial images and the rotational center of the mirror, and which passes through the viewpoint of the camera.
The invention according to the fifteenth aspect comprises an image input apparatus for inputting a wide-viewfield composite image by photographing parts of a field to be photographed while altering the photographic range in the field to be photographed of a camera by rotating a mirror which is supported in a rotatable manner before the camera, and compositing a plurality of partial images of the field to be photographed which are obtained, wherein are provided: a mirror movement mechanism for altering the direction of the mirror so as to change the photographic direction of the camera; a mirror control mechanism for directing the mirror movement mechanism so as to change the orientation of the mirror and for transmitting posture data of the mirror to all parts; a first storage mechanism for storing partial images and composite images produced by compositing the partial images; a second storage mechanism for storing attribute data related to partial images and composite images and the mirror posture data from partial image acquisition; a partial image acquisition mechanism for incorporating images photographed by the camera and acquiring partial images from these incorporated images, and storing the partial images in the first storage mechanism, and for storing posture data from the acquisition of the partial images of the mirror received from the mirror control mechanism, and attribute data relating to the partial images and composite images, in the second storage mechanism; and an image compositing mechanism for determining a pixel on a composite image surface corresponding to a point in a projection surface from an intersection point between an image plane formed by the partial images and a straight line, which is parallel to a straight line indicating a reflected light beam resulting from a reflection by the mirror of a straight line connecting a point on the projection surface which is established in space at the time of the compositing of the plurality of partial images and the rotational center of the mirror, and which passes through the viewpoint of the camera, compositing the partial images with reference to the contents of the first and second storage mechanisms, and storing the composite image in the first storage mechanism.
In accordance with the invention according to the fourteenth and fifteenth aspects, it is possible to obtain a satisfactory wide-viewfield composite image in which the effects of differences in viewpoint contained in the partial images are reduced, and an effect is achieved whereby, when determining the pixel values in the composite image by means of arithmetic processing such as linear interpolation or the like, blank pixels, which are a product of rounding errors in the coordinate values indicating the positions of the pixels, are not produced in the composite image.
The invention according to the sixteenth aspect comprises an image input apparatus for inputting a wide-viewfield composite image by photographing parts of a field to be photographed while altering the photographic range in the field to be photographed of a camera by rotating a mirror which is supported in a rotatable manner before the camera, and compositing a plurality of partial images of the field to be photographed which are obtained, wherein are provided: a mirror movement mechanism for altering the direction of the mirror so as to change the photographic direction of the camera; a mirror control mechanism for directing the mirror movement mechanism so as to change the orientation of the mirror and for transmitting posture data of the mirror to all parts; a first storage mechanism for storing partial images and composite images produced by compositing the partial images; a second storage mechanism for storing attribute data related to partial images and composite images and the mirror posture data from partial image acquisition; a partial image acquisition mechanism for incorporating images photographed by the camera and acquiring partial images from these incorporated images, and storing the partial images in the first storage mechanism, and for storing posture data from the acquisition of the partial images of the mirror received from the mirror control mechanism, and attribute data relating to the partial images and composite images, in the second storage mechanism; a positioning mechanism for determining positional relationships between partial images stored in the first storage mechanism, determining positional correction amounts of the composited positions of the partial images stored in the second storage mechanism based on the positional relationships obtained, and storing these in the second storage mechanism; and an image compositing mechanism for determining compositing positions of the partial images using the posture data of the camera at the time of acquisition of the partial images stored in the second storage mechanism and the positional correction amounts of the compositing positions obtained from the positioning mechanism, and, in accordance with the compositing positions obtained, determining a pixel on a composite image surface corresponding to a point in the projection surface from the intersection point between an image plane formed by the partial images and a straight line, which is parallel to a straight line showing a reflected light beam resulting from the reflection by the mirror of a straight line connecting a point in the projection surface established in space and the rotational center of the mirror, and which passes through the viewpoint of the camera, compositing the partial images with reference to the contents of the first and second storage mechanisms, and storing the composite image in the first storage mechanism.
In accordance with the invention according to the sixteenth aspect, it is possible to obtain a satisfactory wide-viewfield composite image in which the effects of differences in viewpoint contained in the partial images are reduced, and an effect is achieved whereby, when determining the pixel values in the composite image by means of arithmetic processing such as linear interpolation or the like, blank pixels, which are a product of rounding errors in the coordinate values indicating the positions of the pixels, are not produced in the composite image. Furthermore, it is possible to reduce the errors of the angle detection mechanism of the mirror which constitutes the mirror movement mechanism, and to produce highly accurate composite images.
Additionally, by appropriately setting the existing range of orientation of the mirror during the acquisition of the partial images, it is possible to omit the angle detection mechanism of the mirror, which comprises the mirror movement mechanism in the structure of the image input apparatus.
The invention according to the seventeenth aspect has as the gist thereof a computer-readable recording medium storing a program for executing the functions of an image input apparatus for inputting wide-viewfield composite images by photographing parts of a field to be photographed while altering the photographic direction by rotating a camera which is supported in a rotatable manner, projecting a plurality of partial images of the field to be photographed which are obtained onto a projection surface, and based on the results of this projection, conducting compositing on a composite image surface, wherein is provided an image compositing mechanism, which employs the rotational center of the camera as the center of projection during the compositing of the plurality of partial images, and which employs, as the projection point of a pixel of a partial image, the intersection point between the projection surface which is established in space and a straight line, which is parallel to a straight line connecting the viewpoint of the camera and the pixel in the partial image in an image plane formed by the plurality of partial images, and which passes through the rotational center of the camera.
In accordance with the invention according to the seventeenth aspect, by reading a program recorded in the recording medium into a computer system and executing this, it is possible to obtain a satisfactory wide-viewfield composite image in which the effects of differences in viewpoint contained in the partial images are reduced, even in the case in which the camera is rotated about a point other than the principal point of the lens.
The invention according to the eighteenth aspect has as the gist thereof a computer-readable recording medium storing a program for executing the functions of an image input apparatus for inputting wide-viewfield composite images by photographing parts of a field to be photographed while altering the photographic direction by rotating a camera which is supported in a rotatable manner, projecting a plurality of partial images of the field to be photographed which are obtained onto a projection surface, and based on the results of this projection, conducting compositing on a composite image surface, wherein are provided: a camera movement mechanism for changing the photographic direction of the camera; a posture control mechanism for directing the camera movement mechanism so as to change the photographic direction of the camera and for transmitting camera posture data to all parts; a first storage mechanism for storing partial images and composite images produced by compositing the partial images; a second storage mechanism for storing attribute data related to partial images and composite images and camera posture data from partial image acquisition; a partial image acquisition mechanism for incorporating images photographed by the camera and acquiring partial images from these incorporated images, and storing the partial images in the first storage mechanism, and for storing posture data from the acquisition of the partial images of the camera received from the posture control mechanism, and attribute data relating to the partial images and composite images, in the second storage mechanism; and an image compositing mechanism for setting the rotational center of the camera as a center of projection, determining a projection point so that an intersection point between a projection surface established in space and a straight line, which is parallel with a straight line connecting the viewpoint of the camera and a pixel in a partial image in an image plane formed by the plurality of partial images, and which passes through the rotational center of the camera, is employed as the projection point of the pixel in the partial image, compositing the partial images with reference to the contents of the first and second storage mechanisms, and storing a composite image in the first storage mechanism.
In accordance with the invention according to the eighteenth aspect, by reading a program recorded in the recording medium into a computer system and executing this, it is possible to obtain a satisfactory wide-viewfield composite image in which the effects of differences in viewpoint contained in the partial images are reduced, even in the case in which the camera is rotated about a point other than the principal point of the lens.
The invention according to the nineteenth aspect has as the gist thereof a computer-readable recording medium storing a program for executing the functions of an image input apparatus for inputting wide-viewfield composite images by photographing parts of a field to be photographed while altering the photographic direction by rotating a camera which is supported in a rotatable manner, projecting a plurality of partial images of the field to be photographed which are obtained onto a projection surface, and based on the results of this projection, conducting compositing on a composite image surface, wherein are provided: a camera movement mechanism for changing the photographic direction of the camera; a posture control mechanism for directing the camera movement mechanism so as to change the photographic direction of the camera and for transmitting camera posture data to all parts; a first storage mechanism for storing partial images and composite images produced by compositing the partial images; a second storage mechanism for storing attribute data related to partial images and composite images and camera posture data from partial image acquisition; a partial image acquisition mechanism for incorporating images photographed by the camera and acquiring partial images from these incorporated images, and storing the partial images in the first storage mechanism, and for storing posture data from the acquisition of the partial images of the camera received from the posture control mechanism, and attribute data relating to the partial images and composite images, in the second storage mechanism; a positioning mechanism for determining positional relationships between partial images stored in the first storage mechanism, determining positional correction amounts of the composited positions of the partial images stored in the second storage mechanism based on the positional relationships obtained, and storing these in the second storage mechanism; and an image compositing mechanism for determining partial image compositing positions using the camera posture data obtained at the time of acquisition of the partial images stored in the second storage mechanism and the positional correction amounts determined by the positioning mechanism, and in accordance with the compositing positions determined, setting the center of projection to the center of rotation of the camera, and determining the projection point such that the projection point of a pixel in a partial image is set to the intersection point between the projection surface established in space and a straight line, which is parallel to a straight line connecting the viewpoint of the camera and the pixel in the partial image in an image plane formed by the plurality of partial images, and which passes through the rotational center of the camera, and composites the partial images with reference to the contents of the first and second storage mechanisms, and stores a resulting composite image in the first storage mechanism.
In accordance with the invention according to the nineteenth aspect, by reading a program recorded in the recording medium into a computer system and executing this, it is possible to obtain a satisfactory wide-viewfield composite image in which the effects of differences in viewpoint contained in the partial images are reduced, even in the case in which the camera is rotated about a point other than the principal point of the lens, and furthermore, it is possible to reduce the errors of the angle detection mechanism of the camera which constitutes the camera movement mechanism, and to produce highly accurate composite images.
Additionally, by appropriately setting the existing range of orientation of the camera during the acquisition of the partial images, it is possible to omit the angle detection mechanism of the camera, which comprises the camera movement mechanism in the structure of the image input apparatus.
Furthermore, the gist of the invention according to the 20th aspect relates to a computer-readable recording medium which stores a program for executing the functions of an image input apparatus for inputting a wide-viewfield composite image by photographing parts of a field to be photographed while altering the photographic range of a camera in the field to be photographed by rotating a mirror supported in a rotatable manner before the camera, projecting a plurality of partial images of the field to be photographed obtained onto a projection surface, and based on the results of this projection, conducting compositing on a composite image plane, wherein is provided an image compositing mechanism for setting the rotational center of the mirror during the compositing of the plurality of partial images to the center of projection, and setting the intersection point between the projection surface which is established in space and a straight line, which is parallel to a reflection straight line indicating the reflected light beam resulting when a light beam from the viewpoint of the camera and which passes through a pixel of a partial image in an image plane formed by partial images is reflected by the mirror, and which passes through the rotational center of the mirror, as the projection point of the partial image.
Furthermore, in accordance with the invention according to the 20th aspect, a computer-readable recording medium stores a program for executing the functions of an image input apparatus for inputting a wide-viewfield composite image by photographing parts of a field to be photographed while altering the photographic range of a camera in the field to be photographed by rotating a mirror supported in a rotatable manner before the camera, projecting a plurality of partial images of the field to be photographed obtained onto a projection surface, and based on the results of this projection, conducting compositing on a composite image plane, wherein is provided an image compositing mechanism for setting the rotational center of the mirror during the compositing of the plurality of partial images to the center of projection, and setting the intersection point between the projection surface which is established in space and a straight line, which is parallel to a reflection straight line indicating the reflected light beam resulting when a light beam from the viewpoint of the camera and which passes through a pixel of a partial image in an image plane formed by partial images is reflected by the mirror, and which passes through the rotational center of the mirror, as the projection point of the partial image, so that by reading a program recorded in the recording medium into a computer system and executing this, it is possible to obtain a satisfactory wide-viewfield composite image in which the effects of differences in viewpoint contained in the partial images are reduced.
Furthermore, the gist of the invention according to the 21st aspect relates to a computer-readable recording medium which stores a program for executing the functions of an image input apparatus for inputting a wide-viewfield composite image by photographing parts of a field to be photographed while altering the photographic range of a camera in the field to be photographed by rotating a mirror supported in a rotatable manner before the camera, projecting a plurality of partial images of the field to be photographed obtained onto a projection surface, and based on the results of this projection, conducting compositing on a composite image plane, wherein are provided: a mirror movement mechanism for altering the direction of the mirror so as to change the photographic direction of the camera; a mirror control mechanism for directing the mirror movement mechanism so as to change the orientation of the mirror and for transmitting posture data of the mirror to all parts; a first storage mechanism for storing partial images and composite images produced by compositing the partial images; a second storage mechanism for storing attribute data related to partial images and composite images and the mirror posture data from partial image acquisition; a partial image acquisition mechanism for incorporating images photographed by the camera and acquiring partial images from these incorporated images, and storing the partial images in the first storage mechanism, and for storing posture data from the acquisition of the partial images of the mirror received from the mirror control mechanism, and attribute data relating to the partial images and composite images, in the second storage mechanism; and an image compositing mechanism which uses the rotational center of the mirror as the center of projection during the compositing of the plurality of partial images, and which determines a projection point such that the intersection point between the projection surface which is established in space and a straight line, which is parallel to a reflection straight line indicating a reflected beam of light resulting from the reflection by the mirror of a light beam from the viewpoint of the camera which passes through a pixel of a partial image in an image plane formed by the partial images, and which passes through the rotational center of the mirror, is employed as the projection point of the pixel of the partial image, and which composites the partial images with reference to the contents of the first and second storage mechanisms, and stores a composite image in the first storage mechanism.
In accordance with the invention according to the 21st aspect, by reading a program recorded in the recording medium into a computer system and executing this, it is possible to obtain a satisfactory wide-viewfield composite image in which the effects of differences in viewpoint contained in the partial images are reduced.
Furthermore, the gist of the invention according to the 22nd aspect relates to a computer-readable recording medium which stores a program for executing the functions of an image input apparatus for inputting a wide-viewfield composite image by photographing parts of a field to be photographed while altering the photographic range of a camera in the field to be photographed by rotating a mirror supported in a rotatable manner before the camera, projecting a plurality of partial images of the field to be photographed obtained onto a projection surface, and based on the results of this projection, conducting compositing on a composite image plane, wherein are provided: a mirror movement mechanism for altering the direction of the mirror so as to change the photographic direction of the camera; a mirror control mechanism for directing the mirror movement mechanism so as to change the orientation of the mirror and for transmitting posture data of the mirror to all parts; a first storage mechanism for storing partial images and composite images produced by compositing the partial images; a second storage mechanism for storing attribute data related to partial images and composite images and the mirror posture data from partial image acquisition; a partial image acquisition mechanism for incorporating images photographed by the camera and acquiring partial images from these incorporated images, and storing the partial images in the first storage mechanism, and for storing posture data from the acquisition of the partial images of the mirror received from the mirror control mechanism, and attribute data relating to the partial images and composite images, in the second storage mechanism; a positioning mechanism for determining positional relationships between partial images stored in the first storage mechanism, determining positional correction amounts of the composited positions of the partial images stored in the second storage mechanism based on the positional relationships obtained, and storing these in the second storage mechanism; and an image compositing mechanism for determining compositing positions of the partial images using the posture data of the camera at the time of acquisition of the partial images stored in the second storage mechanism and the positional correction amounts of the compositing positions obtained from the positioning mechanism, and, in accordance with the compositing positions obtained, setting the rotational center of the mirror as the center of projection during the compositing of the plurality of partial images, determining a projection point such that the intersection point between the projection surface which is established in space and a straight line, which is parallel to a reflection straight line indicating a reflected beam of light resulting from the reflection by the mirror of a light beam from the viewpoint of the camera which passes through a pixel of a partial image in an image plane formed by the partial images, and which passes through the rotational center of the mirror, is employed as the projection point of the pixel of the partial image, compositing the partial images with reference to the contents of the first and second storage mechanisms, and storing a composite image in the first storage mechanism.
In accordance with the invention according to the 22nd aspect, by reading a program recorded in the recording medium into a computer system and executing this, it is possible to obtain a satisfactory wide-viewfield composite image in which the effects of differences in viewpoint contained in the partial images are reduced, and furthermore, it is possible to reduce the errors of the angle detection mechanism of the mirror which constitutes the mirror movement mechanism, and to produce highly accurate composite images.
Additionally, by appropriately setting the existing range of orientation of the mirror during the acquisition of the partial images, it is possible to omit the angle detection mechanism of the mirror, which comprises the mirror movement mechanism in the structure of the image input apparatus.
Furthermore, the gist of the invention according to the 23rd aspect relates to a computer-readable recording medium for storing a program for executing the functions of an image input apparatus for inputting a wide-viewfield composite image by photographing parts of a field to be photographed while altering the photographic direction by rotating a camera supported in a rotatable manner, and compositing a plurality of partial images of the field to be photographed which are obtained, wherein is provided an image compositing mechanism which determines a pixel on a composite image surface corresponding to a point in a projection surface from the intersection point between an image plane formed by the plurality of partial images and a straight line, which is parallel to a straight line connecting the point in the projection surface established in space during the compositing of the plurality of partial images and the rotational center of the camera, and which passes through the viewpoint of the camera.
In accordance with the invention as disclosed in the 23rd aspect, by reading a program recorded in the recording medium into a computer system and executing this, it is possible to obtain a satisfactory wide-viewfield composite image in which the effects of differences in viewpoint contained in the partial images are reduced, even in the case in which the camera is rotated about a point other than the principal point of the lens, and furthermore, an effect is achieved whereby, when determining the pixel values in the composite image by means of arithmetic processing such as linear interpolation or the like, blank pixels, which are a product of rounding errors in the coordinate values indicating the positions of the pixels, are not produced in the composite image.
Furthermore, the gist of the invention according to the 24th aspect relates to a computer-readable recording medium for storing a program for executing the functions of an image input apparatus for inputting a wide-viewfield composite image by photographing parts of a field to be photographed while altering the photographic direction by rotating a camera supported in a rotatable manner, and compositing a plurality of partial images of the field to be photographed which are obtained, wherein are provided: a camera movement mechanism for changing the photographic direction of the camera; a posture control mechanism for directing the camera movement mechanism so as to change the photographic direction of the camera and for transmitting camera posture data to all parts; a first storage mechanism for storing partial images and composite images produced by compositing the partial images; a second storage mechanism for storing attribute data related to partial images and composite images and camera posture data from partial image acquisition; a partial image acquisition mechanism for incorporating images photographed by the camera and acquiring partial images from these incorporated images, and storing the partial images in the first storage mechanism, and for storing posture data from the acquisition of the partial images of the camera received from the posture control mechanism, and attribute data relating to the partial images and composite images, in the second storage mechanism; and an image compositing mechanism for determining a pixel on a composite image surface corresponding to a point on the projection surface from the intersection point between an image plane formed by the plurality of partial images obtained and a straight line, which is parallel to a straight line connecting a point in the projection surface established in space at the time of the compositing of the plurality of partial images with the rotational center of the camera, and which passes through the viewpoint of the camera, compositing the partial images with reference to the contents of the first and second storage mechanisms, and storing the composite image in the first storage mechanism.
In accordance with the invention according to the 24th aspect, by reading a program recorded in the recording medium into a computer system and executing this, it is possible to obtain a satisfactory wide-viewfield composite image in which the effects of differences in viewpoint contained in the partial images are reduced, even in the case in which the camera is rotated about a point other than the principal point of the lens, and furthermore, an effect is achieved whereby, when determining the pixel values in the composite image by means of arithmetic processing such as linear interpolation or the like, blank pixels, which are a product of rounding errors in the coordinate values indicating the positions of the pixels, are not produced in the composite image.
Furthermore, the gist of the invention according to the 25th aspect relates to a computer-readable recording medium for storing a program for executing the functions of an image input apparatus for inputting a wide-viewfield composite image by photographing parts of a field to be photographed while altering the photographic direction by rotating a camera supported in a rotatable manner, and compositing a plurality of partial images of the field to be photographed which are obtained, wherein are provided: a camera movement mechanism for changing the photographic direction of the camera; a posture control mechanism for directing the camera movement mechanism so as to change the photographic direction of the camera and for transmitting camera posture data to all parts; a first storage mechanism for storing partial images and composite images produced by compositing the partial images; a second storage mechanism for storing attribute data related to partial images and composite images and camera posture data from partial image acquisition; a partial image acquisition mechanism for incorporating images photographed by the camera and acquiring partial images from these incorporated images, and storing the partial images in the first storage mechanism, and for storing posture data from the acquisition of the partial images of the camera received from the posture control mechanism, and attribute data relating to the partial images and composite images, in the second storage mechanism; a positioning mechanism for determining positional relationships between partial images stored in the first storage mechanism, determining positional correction amounts of the composited positions of the partial images stored in the second storage mechanism based on the positional relationships obtained, and storing these in the second storage mechanism; and an image compositing mechanism for determining compositing positions of the partial images using the posture data of the camera at the time of acquisition of the partial images stored in the second storage mechanism and the positional correction amounts of the compositing positions obtained from the positioning mechanism, and, in accordance with the compositing positions obtained, determining a pixel on a composite image surface corresponding to a point on the projection surface from the intersection point between an image plane formed by the plurality of partial images obtained and a straight line, which is parallel to a straight line connecting a point in the projection surface established in space with the rotational center of the camera, and which passes through the viewpoint of the camera, compositing the partial images with reference to the contents of the first and second storage mechanisms, and storing the composite image in the first storage mechanism.
In accordance with the invention according to the 25th aspect, by reading a program recorded in the recording medium into a computer system and executing this, it is possible to obtain a satisfactory wide-viewfield composite image in which the effects of differences in viewpoint contained in the partial images are reduced, even in the case in which the camera is rotated about a point other than the principal point of the lens, and furthermore, an effect is achieved whereby, when determining the pixel values in the composite image by means of arithmetic processing such as linear interpolation or the like, blank pixels, which are a product of rounding errors in the coordinate values indicating the positions of the pixels, are not produced in the composite image.
Furthermore, it is possible to reduce the errors of the angle detection mechanism of the camera which constitutes the camera movement mechanism, and to produce highly accurate composite images.
Additionally, by appropriately setting the existing range of orientation of the camera during the acquisition of the partial images, it is possible to omit the angle detection mechanism of the camera, which comprises the camera movement mechanism in the structure of the image input apparatus.
Furthermore, the gist of the invention according to the 26th aspect relates to a computer-readable recording medium which stores a program for realizing the functions of an image input apparatus for inputting a wide-viewfield composite image by photographing parts of a field to be photographed while altering the photographic range of a camera in the field to be photographed by rotating a mirror supported in a rotatable manner before the camera, and compositing a plurality of partial images of the field to be photographed which are obtained, wherein is provided an image compositing mechanism for determining a pixel on a composite image surface corresponding to a point in a projection surface from the intersection point between an image plane formed by the partial images and a straight line, which is parallel to a straight line indicating the reflected light beam resulting from the reflection of a straight line connecting the point in the projection surface which is established in space during the compositing of the plurality of partial images and the rotational center of the mirror, and passes through the viewpoint of the camera.
In accordance with the invention according to the 26th aspect, by reading a program recorded in the recording medium into a computer system and executing this, it is possible to obtain a satisfactory wide-viewfield composite image in which the effects of differences in viewpoint contained in the partial images are reduced, and furthermore, an effect is achieved whereby, when determining the pixel values in the composite image by means of arithmetic processing such as linear interpolation or the like, blank pixels, which are a product of rounding errors in the coordinate values indicating the positions of the pixels, are not produced in the composite image.
Furthermore, the gist of the invention according to the 27th aspect relates to a computer-readable recording medium which stores a program for realizing the functions of an image input apparatus for inputting a wide-viewfield composite image by photographing parts of a field to be photographed while altering the photographic range of a camera in the field to be photographed by rotating a mirror supported in a rotatable manner before the camera, and compositing a plurality of partial images of the field to be photographed which are obtained, wherein are provided: a mirror movement mechanism for altering the direction of the mirror so as to change the photographic direction of the camera; a mirror control mechanism for directing the mirror movement mechanism so as to change the orientation of the mirror and for transmitting posture data of the mirror to all parts; a first storage mechanism for storing partial images and composite images produced by compositing the partial images; a second storage mechanism for storing attribute data related to partial images and composite images and the mirror posture data from partial image acquisition; a partial image acquisition mechanism for incorporating images photographed by the camera and acquiring partial images from these incorporated images, and storing the partial images in the first storage mechanism, and for storing posture data from the acquisition of the partial images of the mirror received from the mirror control mechanism, and attribute data relating to the partial images and composite images, in the second storage mechanism; and an image compositing mechanism for determining a pixel on a composite image surface corresponding to a point in a projection surface from an intersection point between an image plane formed by the partial images and a straight line, which is parallel to a straight line indicating a reflected light beam resulting from a reflection by the mirror of a straight line connecting a point on the projection surface which is established in space at the time of the compositing of the plurality of partial images and the rotational center of the mirror, and which passes through the viewpoint of the camera, compositing the partial images with reference to the contents of the first and second storage mechanisms, and storing the composite image in the first storage mechanism.
In accordance with the invention according to the 27th aspect, by reading a program recorded in the recording medium into a computer system and executing this, it is possible to obtain a satisfactory wide-viewfield composite image in which the effects of differences in viewpoint contained in the partial images are reduced, and furthermore, an effect is achieved whereby, when determining the pixel values in the composite image by means of arithmetic processing such as linear interpolation or the like, blank pixels, which are a product of rounding errors in the coordinate values indicating the positions of the pixels, are not produced in the composite image.
Furthermore, the gist of the invention according to the 28th aspect relates to a computer-readable recording medium which stores a program for realizing the functions of an image input apparatus for inputting a wide-viewfield composite image by photographing parts of a field to be photographed while altering the photographic range of a camera in the field to be photographed by rotating a mirror supported in a rotatable manner before the camera, and compositing a plurality of partial images of the field to be photographed which are obtained, wherein are provided: a mirror movement mechanism for altering the direction of the mirror so as to change the photographic direction of the camera; a mirror control mechanism for directing the mirror movement mechanism so as to change the orientation of the mirror and for transmitting posture data of the mirror to all parts; a first storage mechanism for storing partial images and composite images produced by compositing the partial images; a second storage mechanism for storing attribute data related to partial images and composite images and the mirror posture data from partial image acquisition; a partial image acquisition mechanism for incorporating images photographed by the camera and acquiring partial images from these incorporated images, and storing the partial images in the first storage mechanism, and for storing posture data from the acquisition of the partial images of the mirror received from the mirror control mechanism, and attribute data relating to the partial images and composite images, in the second storage mechanism; a positioning mechanism for determining positional relationships between partial images stored in the first storage mechanism, determining positional correction amounts of the composited positions of the partial images stored in the second storage mechanism based on the positional relationships obtained, and storing these in the second storage mechanism; and an image compositing mechanism for determining compositing positions of the partial images using the posture data of the camera at the time of acquisition of the partial images stored in the second storage mechanism and the positional correction amounts of the compositing positions obtained from the positioning mechanism, and, in accordance with the compositing positions obtained, determining a pixel on a composite image surface corresponding to a point in a projection surface from an intersection point between an image plane formed by the partial images and a straight line, which is parallel to a straight line indicating a reflected light beam resulting from a reflection by the mirror of a straight line connecting a point on the projection surface which is established in space at the time of the compositing of the plurality of partial images and the rotational center of the mirror, and which passes through the viewpoint of the camera, compositing the partial images with reference to the contents of the first and second storage mechanisms, and storing the composite image in the first storage mechanism.
In accordance with the invention according to the 28th aspect, by reading a program recorded in the recording medium into a computer system and executing this, it is possible to obtain a satisfactory wide-viewfield composite image in which the effects of differences in viewpoint contained in the partial images are reduced, and furthermore, an effect is achieved whereby, when determining the pixel values in the composite image by means of arithmetic processing such as linear interpolation or the like, blank pixels, which are a product of rounding errors in the coordinate values indicating the positions of the pixels, are not produced in the composite image.
Furthermore, it is possible to reduce the errors of the angle detection mechanism of the mirror which constitutes the mirror movement mechanism, and to produce highly accurate composite images.
Additionally, by appropriately setting the existing range of orientation of the mirror during the acquisition of the partial images, it is possible to omit the angle detection mechanism of the mirror, which comprises the mirror movement mechanism in the structure of the image input apparatus.
In order to achieve the second object described above, the invention according to the 29th aspect comprises an image input apparatus which is provided with a camera which photographs a field to be photographed, a mirror which is disposed in front of the camera in order to alter the optical path of the reflected light from the field to be photographed in the direction of the camera lens of the camera, a mirror rotation mechanism for rotating the mirror about the optical axis of the camera, and a processing mechanism for compositing a plurality of images inputted from the camera in accordance with the rotation of the mirror and producing a wide view angle image, wherein the mirror is a triangular columnar mirror in which two of the side surfaces are reflective surfaces, the camera takes photographs using reflected light from the objects to be photographed, the optical path of which is changed by the two reflective surfaces of the mirror, and the processing mechanism separates a plurality of images photographed by the camera in accordance with the reflective surfaces of the mirror and conducts independent image composition.
Furthermore, the invention according to the 30th aspect comprises the image input apparatus as disclosed in the 29th aspect, wherein the processing mechanism is provided with: an image acquisition mechanism which acquires images photographed by the camera and digitizes these, an image separation mechanism which separates the images acquired and digitized by the image acquisition mechanism and separates these according to the two reflective surfaces of the mirror, a first image compositing mechanism which composites the images separated by the image separation mechanism in accordance with a first reflective surface among the two reflective surfaces of the mirror, a second image compositing mechanism, which composites images separated by the image separation mechanism in accordance with a second reflective surface among the two reflective surfaces of the mirror, and an image updating mechanism, which conducts updating by overwriting images composited by the first and second image compositing mechanisms onto a wide-viewfield image.
In accordance with the invention according to the 29th and 30th aspects, it is possible to simultaneously photograph images in two directions and to conduct composition and updating, and for that reason, it becomes possible to input wide-viewfield images at high speed when compositing a plurality of partial images acquired by rotating a mirror in front of a camera.
The invention according to the 31st aspect is provided with: first and second cameras which are disposed so that the optical axes of the camera lens thereof are in agreement and they are in mutual opposition and which photograph a field to be photographed, a double-sided mirror, which is disposed between the first and second cameras and which alters the optical path of light reflected from the field to be photographed in the direction of the camera lens of the first and second camera, a mirror rotation mechanism for rotating the double-sided mirror about the optical axes of the first and second cameras, and a processing mechanism for compositing a plurality of images inputted from the first and second cameras in accordance with the rotation of the mirror and producing a wide view angle image.
Furthermore, the invention according to the 32nd aspect comprises the image input apparatus as disclosed in claim 31, wherein the processing mechanism comprises: a first image acquisition mechanism for acquiring images photographed by the first camera which photographs reflected light from a first reflective surface of the double-sided mirror, and digitizes these images, a second image acquisition mechanism for acquiring images photographed by the second camera which photographs light reflected from a second reflective surface of the double-sided mirror and digitizes these images, a first image compositing mechanism for compositing images digitized by the first image acquisition mechanism, a second image compositing mechanism for compositing images digitized by the second image acquisition mechanism, and an image updating mechanism for conducting updating by overwriting images composited by the first and second image compositing mechanisms onto a wide-viewfield image.
In accordance with the invention according to the 31st and 32nd aspects, it is possible to input wide-viewfield images at high speed when compositing a plurality of partial images acquired by rotating a mirror before cameras, and it is possible to produce wide-viewfield images having higher resolution than that of wide-viewfield images obtained by the image input apparatus in accordance with the invention according to the 29th and 30th aspects.